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1 

DATA MULTIPLEXING APPARATUS AND METHOD THEREOF 
BACKGROUND OP TOE INVENTION 

The present invention relates to an apparatus 
and a method for multiplexing video data, audio data and 
5 additional data such as subtitles and generating a 

transport stream for transmission, wherein the video data 
and audio data may be compressed and encoded according to 
a MPEG standard or the like. 

In digital television broadcasting systems, 

10 audio data and video data may be processed for particular 
applications such as movies or music promotions* More 
specifically, in such digital television broadcasting 
systems, data may be processed into data suitable for a 
so-called surround system, audio data may be processed so 

15 as to presented in several languages, or audio data, 

video data and additional data may be multiplexed so as 
to generate a transport stream wherein a dialogue may be 
superimposed in a number of languages, and broadcasted to 
viewers . 

20 OBJECTS AND SUMMARY OF THE INVENTION 

An object of the present invention is to 
provide a data multiplexer and a method thereof for 
enabling audio data, video data and additional data to be 
multiplexed in a digital television broadcasting system 

25 or the like. 

Another object of the present invention is to 
provide a data multiplexer and a method as aforesaid 
wherein the data may be multiplexed at a relatively high 
speed and wherein changes in the type of the additional 

30 data and in processing may be readily accommodated. 

According to an aspect of the present 
invention, a data multiplexing apparatus is provided 
which comprises a data size detecting device for 
detecting data sizes of input video data and audio data, 

35 a buffer device for buffering the input video data and 
audio data, a device for formulating a multiplexing plan 
for the input video data and audio data in response to 
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the detected data sizes and for controlling the 
multiplexing of the video data and audio data in response 
to the multiplexing plan so as to obtain a multiplexed 
data stream, a bus for connecting the data size detecting 
5 means and the controlling device, a data supply line for 
supplying the input video data and audio data to the 
buffer device, and a data output line for outputting the 
multiplexed data stream to an external apparatus. The 
bus is separate from the data supply line means and the 
10 data output line means so that the video data and audio 
data do not pass through the bus. 

Preferably, the controlling device may include 
a computer, the bus may include a computer bus connected 
to the computer. Further, the present data multiplexer 
15 may also include an encoding device (s) for encoding the 
video data, the audio data and first additional data and 
for supplying the encoded data to the data size detecting 
device and the buffer device. Furthermore, the present 
data multiplexer may also include a data receiving device 
20 for receiving second additional data from outside for 
supply to a data selecting device. 

The present data multiplexer may multiplex 
video data, audio data and first additional data such as 
subtitles for use in a digital television broadcasting 
25 system and may further multiplex second additional data 
(private data) such as superimposed dialogue into 
multiplexed data so as to generate a transport data 
stream. In such data multiplexer, a multiplexing system 
for directly multiplexing video data and other data (such 
30 as audio data and so forth) and a control system, 

separate from the multiplexing system, for controlling 
the multiplexing may be provided and arranged such that 
the video and other data to be multiplexed and data 
associated with multiplexing control may be respectively 
35 processed in the corresponding system. 

in the present data multiplexer, the data size 
determining or counting device may be arranged within the 
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control system and may count video data, audio data and 
first additional data after such data has been compressed 
and encoded according to a MPEG standard or the like so 
as to obtain data size information thereof. The buffer 
5 device may be arranged within the multiplexing system and 
may include FIFO (first in first out) memories or the 
like for buffering the video data, the audio data, and 
the first additional data. First and second data 
multiplexing portions may be included within the 

10 multiplexing system, wherein the first data multiplexing 
portion may select and multiplex the buffered video data, 
audio data, and/or first additional data, and the second 
data multiplexing portion may select any of the video 
data, the audio data and the first additional data which 

15 are selected and multiplexed by the first data 

multiplexing portion or may select the second additional 
data such as superimposed dialogue data under the control 
of the control system and may multiplex any of the video 
data, the audio data and the first additional data which 

20 are multiplexed and the second additional data. The 
controlling device may be arranged within the control 
system and may formulate a plan for multiplexing the 
video data, the audio data, the first additional data and 
the second additional data, timing adjustment, scheduling 

25 or the like based on the data size of the video data, the 
audio data and the additional data as determined or 
counted by the data size counting device. The 
controlling device may also control the selection by the 
first and second data multiplexing portions based on the 

30 multiplexing plan so as to control the multiplexing of 
the data in the first and second data multiplexing 
portions. 



invention, a data multiplexing method is provided which 
35 comprises the steps of receiving video data, audio data, 
first additional data, and second additional data; 
obtaining data sizes of the received video data, audio 



According to another aspect of the present 
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data and first additional data; buffering the received 
video data, audio data and first additional data; 
selecting the buffered video data, audio data or first 
additional data; selecting the received second additional 
5 data or the selected one of the buffered video data, 

audio data, and first additional data; formulating a plan 
for multiplexing the video data, the audio data, the 
first additional data, and the second additional data 
based on the data size of the video data, the audio data 
10 and the additional data; and controlling the selection of 
the buffered video data, audio data or first additional 
data and the selection of the second additional data or 
the selected one of the buffered video data, audio data, 
and first additional data based on the multiplexing plan 
15 so as to multiplex the video data, the audio data, the 
first additional data, and the second additional data. 

Other objects, features and advantages 
according to the present invention will become apparent 
from the following detailed description of illustrated 
20 embodiments when read in connection with the accompanying 
drawings in which corresponding components are identified 
by the same reference numerals. 

fP T ??r T^BBCHTPTION OF THE PRAWiyOB 
Fig. 1 is a diagram of a data multiplexer 
25 according to an embodiment of the present invention; 

Fig. 2 is a diagram of the data multiplexer 
according to an embodiment of the present invention; 

Fig. 3 is a diagram of a video encoder in the 

data multiplexer of Fig. 2; 
30 Fig. 4 is a diagram of an audio encoder in the 

data multiplexer of Fig. 2; and 

Fig. 5 is a flowchart to which reference will 
be made in explaining the processing performed by a CPU 
in the data multiplexer of Fig. 2. 
35 rr ^ T y n.TON O f "UK »««™*T> MWOMMBTO 

Fig. 1 illustrates a data multiplexer l. As 
shown therein, such data multiplexer may include a video 
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encoder 20, an audio encoder 24, a subtitle encoder 28, 
and a multiplexer 10 having a buffer memory 100 and a 
transport stream maker 102 . The data multiplexer 1 may 
receive and encode a video stream, an audio stream, and a 
5 subtitle stream (for movie or music promotion) and may 
multiplex the same into one stream so as to generate a 
transport stream, as here inbe low more fully described. 

More specifically, the video encoder 20 may 
receive video data from an external device (not shown) 

10 such as an editing device and so forth and may compress 
and encode the received video data according to a 
predetermined standard (such as an MPEG 2 standard or the 
like) and output the obtained data as a video stream to 
the buffer memory 100. The audio encoder 24 may receive 

15 audio data from an external device (not shown) and may 
compress and encode the received audio data according to 
a predetermined standard (such as an MPE62 standard or 
the like) and output the obtained data as an audio stream 
to the buffer memory 100. The subtitle encoder 28 may 

20 receive subtitle data from an external generating 
subtitle data device (not shown) and may encode the 
received subtitle data by a predetermined process (such 
as a linear quantization process and a fixed-length 
encoding process) and output the obtained result as a 

25 subtitle stream to the buffer memory 100. 

The buffer memory 100 buffers the video stream, 
the audio stream and the subtitle stream respectively 
supplied from the video encoder 20, the audio encoder 24 
and the subtitle encoder 28, and supplies an output to 

30 the transport stream maker 102. The transport stream 

maker 102, which may include a high-speed computer having 
a microprocessor (CPU) or the like, may obtain or read 
out the video stream, the audio stream, and the subtitle 
stream (each of which may be referred to as an elementary 

35 stream) from the buffer memory 100, may perform 
scheduling, and may add a header or the like in 
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accordance with a transmission format so as to generate a 
transport stream for output therefrom. 

In the data multiplexer 1, the buffer memory 
100 temporarily stores all the elementary streams therein 
and the CPU of the transport stream maker 102 performs 
processing such as scheduling and addition of a header to 
generate a transport stream. However, in performing such 
processing, the CPU may access each word (byte) in the 
elementary streams through a data bus in such a manner 
that a so-called bus neck or the like may be generated in 
the data bus of the transport stream maker 102 which may 
hinder or prevent high-speed processing. Accordingly, 
such processing manner may not be acceptable for forming 

a transport stream. 

A data multiplexer 2 will now be described with 
reference to Figs. 2-5 which may overcome the above- 
described problems of the data multiplexer 1. In such 
multiplexer 2, by separating a control system for 
performing scheduling for multiplexing the elementary 
streams from a multiplexing system for directly 
multiplexing the elementary streams, high-speed 
production of a transport stream may be obtained, as 
hereinbelow more fully described. 

As shown in rig. 2, the data multiplexer 2 
25 generally includes the video encoder 20, the audio 
encoder 24, the subtitle encoder 28, a multiplexing 
system 12, and a control system 42. The multiplexing 
system 12 may include FIFO memories 32a, 32b, and 32c, a 
RAM 430, a first switch circuit 34, a second switch 
30 circuit 36, a FIFO memory 38, and a SCSI (small computer 
system interface) circuit 40, which may be connected as 
shown in Fig. 2. The control system 42 may include data 
size counting interface circuits 30a, 30b, and 30c, an 
ethernet interface (ENIF) circuit 420, a serial interface 
35 (SIF) circuit 422, a CPU 424, a processing RAM 426, and a 
control data RAM 428, which may be coupled together 
through a CPU bus as shown in Fig. 2. 



20 
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The video encoder 20 may include adding 
circuits 202 and 212, a discrete cosine transform (DCT) 
circuit 204, a quantizing circuit 206, an inverse 
quantizing circuit 208, an inverse DCT circuit 210, a 
5 frame memory circuit 216, a variable-length encoding 

(VLC) circuit 218, a bit rate control circuit 220, and a 
variable-length encoding (VLC) buffer 222, which may be 
arranged as shown in Fig. 3. The video encoder 20 is 
adapted to receive an input video data (Video IN) from an 

10 external device and to compress and encode the same 
according to a predetermined standard, such as a MPEG 2 
standard or the like, and to generate a video stream 
(MPEG Video stream) for supply to the data size IF 30a 
and the FIFO memory 32a (Fig. 2). A further description 

15 of such video encoder may be obtained from U.S. 

application Serial No. , filed on February 

26, 1997 entitled "Image Data Processing Method and 
Apparatus of Same 19 , which is hereby incorporated by 
reference. 

20 The audio encoder 24 may include a sub-band 

analyzing filter bank (SAFB) 240, a linear quantizing 
circuit 242, a bit compression circuit 244, a fast 
fourier transfer (FFT) circuit 246, a psychological 
auditory model 248, a dynamic bit assignment circuit 250, 

25 a scale factor selection information storing circuit 252, 
a scale factor extracting circuit 254, a side information 
encoding circuit 256, and a bit stream generating circuit 
258, as shown in Fig* 4. The audio encoder 24 is adapted 
to receive audio data (Audio Input) from an external 

30 device and to compress and encode the same according to a 
predetermined standard, such as a MPEG2 standard or the 
like, and to generate an audio stream (MPEG Audio stream) 
for supply to the data size IF 30b and the FIFO memory 
32b (Fig. 2). A further description of such audio 

35 encoder may be obtained from an article entitled " ISO- 
MPEG- 1 Audio: A Generic Standard for Coding of High- 
Quality Digital Audio", by K. Brandenburg et al., J. 
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Audio Eng. Soc, Vol. 42, No. 10, October 1994, pages 
780-792, which is hereby incorporated by reference. 

The subtitle encoder 28 is adapted to receive 
additional data such as subtitle data from an external 
5 device and to encode the same, in a manner similar to 
that previously described, and to generate a subtitle 
stream for supply to the data size IF 30c and the FIFO 
memory 32c (Fig* 2) . 

Further, encoded data (such as subtitle data, 
10 closed-caption data, and so forth) may be supplied from 
an external device. For example, encoded subtitle data 
may be supplied through a LAN (local area network) such 
as the Ethernet to the ENIF circuit 420 or encoded 
subtitle data may be supplied through a serial line to 
15 the serial interface 422, and such received encoded 

subtitle data may be supplied through the CPU bus to the 
RAM 430 for storage therein. Since the amount of encoded 
subtitle data is relatively small, as compared to the 
amount of encoded video data and/or encoded audio data, 
20 supplying the encoded subtitle data on the CPU bus may 
not adversely affect the CPU bus traffic. In this 
situation, the ENIF circuit 420 or the serial interface 
422 may detect information such as the header and/or size 
information (which may be written in the header) in the 
25 encoded subtitle data, and the detected information may 
be supplied through the CPU bus to the CPU 424. 

In the multiplexing system 12 (Fig. 2), the 
FIFO memories 32a, 32b and 32c respectively buffer the 
video stream, the audio stream and the subtitle stream 
30 received from the video encoder 20, the audio encoder 24 
and the subtitle encoder 28 and supply output streams to 
input terminals a, b and c of the switch circuit 34. 
Additionally, the encoded subtitle data stored in the RAM 
430 may be supplied to an input terminal d of the 
35 switching circuit 34 under the control of the CPU 424. 

In accordance with a control signal which may be supplied 
from the CPU 24, the switch circuit 34 selects any of the 
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input terminals a, b, c and d and supplies the respective 
elementary stream supplied to the selected input terminal 
to an input terminal b of the switch circuit 36. As a 
result, a multiplexed signal or stream may be formed and 
5 supplied to the input terminal b of the switch circuit 
36. The switch circuit 34 may select none of the input 
terminals a, b, c and d and, as such, output a 
predetermined amount of blank data (for example, 
consecutive logic values of 1 or 0) when no elementary 

10 stream is supplied to any of the input terminals or when 
a so-called stuffing process is being performed. 

The RAM 26 may supply a signal or additional 
data stream, which may be representative of a header, to 
an input terminal a of the switch circuit 36. In 

15 accordance with a control signal which may be supplied by 
the CPU 424, the switch circuit 36 may select any of the 
input terminals a and b so as to multiplex the elementary 
streams supplied from the switch circuit 34 to the input 
terminal b of the switch circuit 36 and the additional 

20 data stream (header information) supplied from the 

processing RAM 426 to the input terminal a of the switch 
circuit 36. A multiplexed or selected data stream is 
supplied from the switch circuit 36 to the FIFO memory 38 
and the SCSIIF circuit 40. 

25 The FIFO memory 38 buffers the multiplexed data 

stream received from the switch circuit 36 and supplies 
an output or a transport stream to an external device 
(not shown) such as a communication line and so forth. 
The SCSIIF circuit 40 receives the multiplexed data 

30 stream from the switch circuit 36 and supplies the same 
to a storage device (not shown) such as a hard disk drive 
(HDD) or a magnetic optical disk drive (MOD) wherein the 
data stream is stored. 

In the control system 42, the data size IFs 

35 30a, 30b, and 30c may count the data size for each frame 
of the video stream, the audio stream, and the subtitle 
stream respectively supplied from the video encoder 20, 



WO 97/35435 



PCT/IB97/00543 



10 

the audio encoder 24, and the subtitle encoder 28 and may 
supply the obtained result to the CPU 42 by way of the 
CPU bus. Such counting of data sizes may be achieved by 
use of counters which may be included in the data size 
5 IFs 30a, 30b, and 30c. Alternatively, data size 

information may be determined by the respective encoder 
and supplied as an output signal therefrom, whereupon 
such data size information signal may be detected and 
supplied to the CPU 424. 

10 The ENIF circuit 420 may receive private data 

through a LAN (not shown) such as Ethernet and may output 
data to the CPU 424 through the CPU bus. The SIF circuit 
422 may receive private data having a serial format from, 
for example, a computer or the like, and may output data 

15 to the CPU 424. Such so-called private data may include 
encoded subtitle data, closed caption data, user data, 

and so forth. 

The CPU 424 may include a microprocessor and a 
ROM for storing programs. Such CPU is adapted to receive 
20 and store the data sizes from the data size IFs 30a, 30b 
and 30c, and data from the ENIF circuit 420 and the SIF 
circuit 422. The CPU is further adapted to formulate a 
multiplexing plan or to control the multiplexing 
operation of the present data multiplexer. That is, the 
25 CPU 424 may control the multiplexing of the elementary 
streams, timing adjustment and scheduling for 
multiplexing based on the data sizes stored in the 
processing RAM 426, and the switching or multiplexing 
operation of the switch circuits 34 and 36 according to 
30 the multiplexing plan by way of the CPU bus. 

Control data for processing the CPU 424 may be 
stored in the control data RAM 428. Additionally, the 
CPU may control or utilize the processing RAM 426 for 
processing, storing, and/ or supplying data. For example, 
35 the CPU 424 may use the processing RAM 426 to generate 
header information, store such header information, and 
output the same to the input terminal a of the switch 
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circuit 36 on the basis of a scheduling procedure or the 
like in accordance with control data stored in the 
control data RAM 428. Alternatively, the header 
information or a portion thereof may be generated on the 
5 basis of private data (such as user data) supplied from 
the ENIF circuit 420 or the SIF circuit 422 and stored in 
the RAM 426. 

An operation of the data multiplexer 2 will now 
be described hereinbelow. 

10 Video data, audio data, and subtitle data are 

respectively supplied to the video encoder 20, the audio 
encoder 24, and the subtitle encoder 28 and encoded 
thereat. The data size of the encoded video stream, 
audio stream and subtitle stream are determined by 

15 counting the data in the data size IFs 30a, 30b and 30c 
and the obtained counted data sizes are supplied to the 
CPU 424. 

On the other hand, when encoded subtitle data 
is supplied through a LAN (such the Ether network) to the 

20 ENIF circuit 420, or when encoded subtitle data is 

supplied through a serial line to the serial interface 
422, the ENIF circuit 420 or the serial interface 422 may 
detect the data size, whereupon the detected data size is 
supplied to the CPU 424 through the CPU bus. In this 

25 situation, the encoded sub-code data may be supplied to 
the RAM 430 and outputted therefrom to the input terminal 
d of the switching circuit 34 under the control of the 
CPU 424. 

Data may be switched or multiplexed by use of 
30 switch circuits 34 and 36 in accordance with control 
signals from the CPU 424. An output from the switch 
circuit 36 is supplied to the FIFO 38 and the SCSIIF 
circuit 40. The FIFO 38 buffers the received data and 
outputs a transport stream which may be supplied over a 
35 communication line or the like. The SCSIIF circuit 40 
transmits the received data to a hard disc drive or the 
like so as to record the same. 
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The processing performing by the CPU 24 will 
now be further described with reference to the flowchart 
of Fig. 5. 

At step S102, the CPU 424 receives the data 
5 sizes of a frame (s) for a video stream, an audio stream 
and a subtitle stream from the data size IF 30a, 30b and 
30c and stores the received data sizes in the processing 
RAM 426. The CPU 424 may receive data sizes from the 
ENIF circuit 420 and the SIF circuit 422, whereupon the 
10 CPU 424 stores such data sizes. Processing then proceeds 
to step S103 wherein the parameters may be initialized by 
the CPU 424. Processing then proceeds to step S104. 

In step S104, the CPU 424 formulates a 
multiplexing plan. That is, a loop iteration number N is 
15 determined. Such loop iteration number N may indicate 
the number of packets which may be inserted into one 
video frame or a duration thereof, in which the size of a 
packet may have a fixed predetermined value such as 188 
bytes. Accordingly, the loop iteration number N may be 
20 determined on the basis of the data rate of the transport 
stream. For example, the loop iteration number N may be 
defined as the data rate (bytes) of the transport 
steam/ 30 frames per second (when a NTSC system is 
utilized) /188 (bytes). The scheduling of the 
25 multiplexing is determined on the basis of the loop 
iteration number N and the received respective data 
sizes. 

Processing then proceeds to step S106, wherein 
a determination is made by the CPU 424 as to whether the 

30 multiplexing plan indicates that a video stream is to be 
out put ted. If such determination is affirmative, 
processing proceeds to step S107. At step S107, the CPU 
424 controls the switch circuit 36 such that header 
information is selected from input terminal a which, in 

35 turn, may be supplied from the RAM 426. Thereafter, 
processing proceeds to step S108, wherein the CPU 424 
controls the switch circuit 34 to select the video stream 
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(which may be obtained from terminal a of switch circuit 
34) and controls the switch circuit 36 to select the data 
stream from the switch circuit 34 (which may be obtained 
from terminal b of switch circuit 36) so that the video 
5 stream is multiplexed to the transport stream. 
Processing then proceeds to step S110. 

If, on the other hand, the determination in 
step S106 is negative, processing proceeds to step S110. 

In step S110, a determination is made by the 

10 CPU 424 as to whether the multiplexing plan indicates 
that an audio stream is to be outputted. If such 
determination is affirmative, processing proceeds to step 
Sill. At step Sill, the CPU 424 controls the switch 
circuit 36 such that header information is selected from 

15 input terminal a which, in turn, may be supplied from the 
RAM 426. Thereafter, processing proceeds to step S112, 
wherein the CPU 424 controls the switch circuit 34 to 
select the audio stream (which may be obtained from 
terminal b of switch circuit 34) and controls the switch 

20 circuit 36 to select the data stream from the switch 
circuit 34 (which may be obtained from terminal b of 
switch circuit 36) so that the audio stream is 
multiplexed to the transport stream. 

If, on the other hand, the determination in 

25 step S110 is negative, processing proceeds to step S114. 

In step S114, a determination is made by the 
CPU 424 as to whether the multiplexing plan (procedure) 
indicates that a subtitle stream is to be outputted from 
the subtitle encoder. If such determination is 

30 affirmative, processing proceeds to step 115. At step 
S115, the CPU 424 controls the switch circuit 36 such 
that header information is selected from input terminal a 
which, in turn, may be supplied from the RAM 426. 
Thereafter, processing proceeds to step S116, wherein the 

35 CPU 424. controls the switch circuit 34 to select the 

subtitle stream (which may be obtained from terminal c of 
switch circuit 34) and controls the switch circuit 3 6 to 



WO 97/35435 



PCT/IB97/00543 



14 

select the data stream from the switch circuit 34 (which 
may be obtained from terminal b of switch circuit 36) so 
that the subtitle stream is multiplexed to the transport 
stream. Processing then proceeds to step SI 18. 
5 If , on the other hand, the determination in 

step S114 is negative, processing proceeds to step S118. 

In step S118, a determination is made by the 
CPU 424 as to whether the multiplexing plan indicates 
that an additional or private data stream (header 
10 information) is to be outputted. If such determination 
is af f irmative, processing proceeds to step S119. At 
step S118, the CPU 424 controls the switch circuit 36 to 
select the additional data stream in order to multiplex 
the additional data stream (header information) to the 
15 transport stream. Processing then proceeds to step S120, 
wherein the CPU 424 controls the switch circuit 34 , the 
switch circuit 36 and the RAM 430 to output the private 
data from the RAM 430 , resulting in the multiplexing of 
the same to the transport stream. Thereafter, processing 
20 then proceeds to step S122. 

If, on the other hand, the determination in 
step S118 is negative, processing proceeds to step S122. 

At step S122, a determination is made by the 
CPU 424 as to whether n»N. If such determination is 
25 affirmative, that is, if n=N, then the processing is 

terminated. However, if such determination is negative, 
that is, if nN, N is incremented by one and processing 
returns to step S106. 

Therefore, in the data multiplexer 2, 
30 information indicative of the size of the video data, 

audio data and/or additional data may be supplied to the 
CPU by way of a CPU or data bus; whereas the video data, 
audio data and/or additional data or elementary stream(s) 
thereof which may have a relatively large amount of data 
35 may not transported to the CPU over the CPU bus. 

instead, the video data, audio data and/or additional 
data may be supplied through a number of switch circuits 
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so as to form a transport or multiplexed stream. As a 
result, the likelihood of a bottleneck or bus neck 
occurring on the CPU bus is greatly reduced so that 
high-speed multiplexing of the elementary stream to a 
5 transport stream may be achieved. 

Further, the subtitle encoder 28 and the 
private data lines may both be utilized to supply 
information. For example, subtitle information may be 
supplied through the subtitle encoder 28 and closed 
10 captioned may be supplied through the private data lines. 
As another example, subtitle information may be supplied 
through the private data lines and other information may 
be supplied through the subtitle encoder 28. As is to be 
appreciated, either of the private data lines or the 
15 encoder 28 may be omitted depending upon the desired 
application or use. 

Additionally, since a multiplexing plan is 
formulated by software processing performed by the CPU 
424 each time the transport stream is produced, the 
20 prevent invention enables the type or number of 

additional data stream (header information) to be added 
to be readily changed as compared to the arrangement 
wherein an elementary stream is multiplexed to the 
transport stream using hardware. 
25 Although the data multiplexer of Fig. 2 was 

described has having a certain number of each type of 
elements, the present data multiplexer is not so limited 
and may have other numbers of such elements. For 
example, none or more than one EN IF circuit 420 or SIF 
30 circuit 422 may be utilized. Further, any number of 
elementary streams may be multiplexed to obtain a 
transport stream. 

Additionally, the data multiplexer 2 may be 
modified so as to obtain a transport stream which may be 
35 desired for a particular application. For example, the 
data multiplexer 2 may be modified so that a data stream 
associated with an editing process or a broadcasting 
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process is multiplexed in place of the subtitle stream to 
obtain the transport stream. 

Thus, as described above, the present data 
multiplexer may enable audio data, video data and 
5 additional data to be multiplexed in a digital television 
broadcasting system or the like at a relatively high 
speed and may enable changes in the kind of additional 
data and in processing to be readily incorporated. 

Although preferred embodiments of the present 
10 invention and modifications thereof have been described 
in detail herein, it is to be understood that this 
invention is not limited to these embodiments and 
modifications, and that other modifications and 
variations may be effected by one skilled in the art 
15 without departing from the spirit and scope of the 
invention as defined by the appended claims. 
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WHAT IB CIA1MBD Ifli 

1* A data multiplexing apparatus comprising: 
data size detecting means for detecting data 

sizes of input video data and audio data; 
5 buffer means for buffering said input video 

data and audio data; 

means for formulating a multiplexing plan for 

said input video data and audio data in response to the 

detected data sizes and for controlling the multiplexing 
10 of said video data and audio data in response to said 

multiplexing plan so as to obtain a multiplexed data 

stream; 

bus means for connecting said data size 
detecting means and said controlling means; 
15 data supply line means for supplying said input 

video data and audio data to said buffer means; and 

data output line means for outputting said 
multiplexed data stream to an external apparatus, 

said bus means being separated from said data 
20 supply line means and said data output line means such 
that said video data and audio data do not pass through 
said bus means. 

2. A data multiplexing apparatus as in claim 
1, wherein said controlling means includes a computer, 

25 and wherein said bus means includes a computer bus 
connected to said computer. 

3. A data multiplexing apparatus as in claim 
1, further comprising encoding means for encoding the 
video data, the audio data, and first additional data for 

30 supply to said data size detecting means and said buffer 
means. 

4. A data multiplexing apparatus as in claim 
1, further comprising data receiving means for receiving 
additional data and for supplying the received additional 

35 data to data selecting means. 5. A data 

multiplexing method comprising the steps of: 
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receiving video data, audio data, first 
additional data, and second additional data; 

obtaining data sizes of the received video 
data, audio data and first additional data; 
5 buffering the received video data, audio data 

and first additional data; 

selecting the buffered video data, audio data 
or first additional data; 

selecting the received second additional data 
10 or the selected one of the buffered video data, audio 
data, and first additional data; 

formulating a plan for multiplexing the video 
data, the audio data, the first additional data, and the 
second additional data based on the data size of the 
15 video data, the audio data and the additional data; and 

controlling the selection of the buffered video 
data, audio data or first additional data and the 
selection of the second additional data or the selected 
one of the buffered video data, audio data, and first 
20 additional data based on the multiplexing plan so as to 
multiplex the video data, the audio data, the first 
additional data, and the second additional data. 

6. A data multiplexing apparatus comprising: 
means for receiving audio data and video data; 
25 means for obtaining size information for said 

audio data and said video data; 

control means for receiving said size 
information from the obtaining means by way of a first 
supply means and for controlling a multiplexing operation 
30 of said audio data and said video data in accordance with 
said size information; and 

means including a second supply means for 
multiplexing said audio data and said video data so as to 
form a multiplexed data stream and for supplying the same 
35 therefrom; 
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said first supply means and said second supply 
means being arranged such that said audio data and said 
video data are not supplied to said computation means. 

7. A data multiplexing apparatus comprising: 
5 means for receiving audio data, video data, and 

additional data and for obtaining size information for 
said audio data, said video data, and said additional 
data; 

control means for receiving said size 
10 information from the obtaining means by way of a data bus 
and for providing control signals in accordance with said 
size information; and 

means for multiplexing said audio data, said 
video data, and said additional data in accordance with 
15 said control signals so as to form a multiplexed data 
stream and for supplying the same therefrom; 

said data bus being arranged such that said 
audio data and said video data are not supplied to said 
control means. 

20 8. A data multiplexing apparatus according to 

claim 7, wherein said additional information includes 
subtitle data. 

9. A data multiplexing apparatus according to 
claim 6, wherein said means for obtaining size 

25 information includes counters for counting said audio 
data, said video data and said additional data. 

10. A data multiplexing method comprising the 

steps of: 

receiving audio data, video data, and 
30 additional data and obtaining size information for said 
audio data, said video data, and said additional data; 

supplying said size information by way of a 
data bus to a central processing unit (CPU) which forms 
control signals in accordance with said size information; 
35 and 
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multiplexing said audio data, said video data, 
and said additional data in accordance with said control 
signals so as to form a multiplexed data stream; 

said data bus being arranged such that said 
5 audio data and said video data are not supplied to said 
control means. 

11. A data multiplexing method according to 
claim 10, wherein said additional information includes 
subtitle data. 

10 12. A data multiplexing method according to 

claim 10, wherein said size information is obtained by 
counting said audio data, said video data and said 
additional data. 
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